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Abstract 

This paper considers a controlled Ito-Levy process where the information available 
to the controller is possibly less than the overall information. All the system coefficients 
and the objective performance functional are allowed to be random, possibly non- 
Markovian. Malliavin calculus is employed to derive a maximum principle for the 
optimal control of such a system where the adjoint process is explicitly expressed. 
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1 Introduction 

Suppose the state process X(t) = XM(t,w); t > 0, u E tt, is a controlled Ito-Levy process 
in K of the form 

dX(t) = b(t, X(t), u(t),u)dt + a(t, X(t),u(t),u)dB(t) 



+ J Rq 9(t, X(t-),u(t-), z, u)N(dt, dz); 
X(0) = x ER. 
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Here R = R - {0}, B{t) = B(t,u), and T](t) = rj(t,uj), given by 
(1.2) r](t)= [ [ zN(ds,dz); t > 0, uj e Q, 



are a 1-dimensional Brownian motion and an independent pure jump Levy martingale, re- 
spectively, on a given filtered probability space (O, T, {T}t>o, P)- Thus 

(1.3) N(dt, dz) := N(dt, dz) - v{dz)dt 

is the compensated jump measure of f]{-), where N(dt,dz) is the jump measure and v(dz) 
the Levy measure of the Levy process r)(-). The process u(t) is our control process, assumed 
to be jF f -adapted and have values in a given open convex set U C M. The coefficients 
6:[0 1 T]xMx[/xn^K,(j:[0 1 T]xMx[/xn^Mande:[0,T]xlx[/xM o xfiare 
given ^-predictable processes. 

We refer to [YZ] and [0S] for more information about stochastic control of Ito diffusions 
and jump diffusions, respectively. Let T > be a given constant. For simplicity, we assume 
that 



(1.4) / z l v{dz) < oo. 

Suppose in addition that we are given a subfiltration 

StQFt, te[0,T\ 

representing the information available to the controller at time t and satisfying the usual 
conditions. For example, we could have 

Sf = J-u_g\+\ t G [0, T], 5 > is a constant, 

meaning that the controller gets a delayed information compared to Tt- 

Let srf = stfg denote a given family of controls, contained in the set of ^-adapted cadlag 
controls u(-) such that (11. II) has a unique strong solution up to time T. Suppose we are given 
a performance functional of the form 



T 



1.5) J(u) = E / f(t,X(t),u(t),u)dt + g(X(T),u) 



where E = Ep denotes expectation with respect to P and / : [0,T] x M X U x Q — > R and 
j:lxO^K are given Tt— adapted processes with 



E 



\f(t,X(t),u(t))\dt+\g(X(T))\ 



< oo for all u G srfg 



The partial information control problem we consider is the following: 
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Problem 1.1 Find $^gR and u* e srfg (if it exists) such that 



(1.6) 



<JV = sup J(u) = J(u*). 



Note that since we allow b, a, 6, f and g to be stochastic processes and also because our 
controls must be ^—adapted, this problem is not of Markovian type and hence cannot be 
solved by dynamic programming. We instead investigate the maximum principle, and derive 
an explicit form for the adjoint process. The approach we employ is Malliavin calculus which 
enables us to express the duality involved via the Malliavin derivative. Our paper is related 
to the recent paper [B0], where a maximum principle for partial information control is 
obtained. However, that paper assumes the existence of a solution of the adjoint equations. 
This is an assumption which often fails in the partial information case. 

We emphasize that our problem should be distinguished from the partial observation 
control problems, where it is assumed that the controls are based on noisy observation 
of the state process. For the latter type of problems, there is a rich literature see e.g. 
[BEK], [B], [KX], [L], [PQ], [T]. Note that the methods and results in the partial observation 
case do not apply to our situation. On the other hand, there are several existing works 
on stochastic maximum principle (either completely or partially observed) where adjoint 
processes are explicitly expressed [BEK], [EK], [L], [T]. However, these works all essentially 
employ stochastic flows technique, over which the Malliavin calculus has an advantage in 
terms of numerical computations (see, e.g., [FLLL]). 

2 A brief review of Malliavin calculus for Levy pro- 
cesses 

In this section we recall the basic definition and properties of Malliavin calculus for Levy 
processes related to this paper, for reader's convenience. 

In view of the Levy-Ito decomposition theorem, which states that any Levy process Y(t) 
with 



with constants a and b, we see that it suffices to deal with Malliavin calculus for £>(•) and 



E[Y 2 {t)} < oo for all t 



can be written 
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for 



v (.) ■= / / zN(ds,dz) 

J0 Jr 



separately. 

A general reference for this presentation is [N], [BDL0P] and [DM0P]. See also the 
forthcoming book [D0P]. 

2.1 Malliavin calculus for B(-) 

A natural starting point is the Wiener-Ito chaos expansion theorem, which states that any 
F G L 2 (jF T , P) can be written 

oo 

(2-1) F = J2Wn) 

n=0 

for a unique sequence of symmetric deterministic functions /„ G L 2 (A n ), where A is Lebesgue 
measure on [0, T] and 

(2.2) Wn)=n\ / •••/ Ufa,--- ,t n )dB(t 1 )dB(t 2 )---dB(t n ) 

Jo Jo Jo 

(the n-times iterated integral of /„ with respect to B(-)) for n = 1,2, . . . and io(/o) — fo 
when /o is a constant. 

Moreover, we have the isometry 



|2 

Il 2 (A")- 



(2-3) ^^ 2 ] = ll^llW) = E n! H/«l 

n=0 

Definition 2.1 (Malliavin derivative D t ) 

Let be the space of all F G L 2 (jF T ,P) such that its chaos expansion (2.1) satisfies 

oo 

(2-4) \\ F \\l[*) := 5Z nn! H^II^ 2 (A") < °°- 

1,2 71=1 

For F G and t G [0, T], we define the Malliavin derivative of F at t (with respect to 
£(•)), AF, by 

oo 

(2.5) D t F = Y t nI n -i(f n (;t)), 

71=1 

where the notation I n _i(f n (-,t)) means that we apply the (n — l)-times iterated integral to 
the first n — 1 variables ti, • • • , t n -\ of fn(ti,h, " • ,t n ) and keep the last variable t n = t as 
a parameter. 
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One can easily check that 

(2.6) E\ f{D t Ffdt\ = f>n!||/„||i 2(A „ ) = \\F\ 

lJ J n =l 

so (t,u) -> A-FM belongs to L 2 (A x P). 

Example 2.2 If F = J Q T f(t)dB(t) with / G L 2 (A) deterministic, then 

D t F = /(*) for a.a.te [0,T]. 

More generally, if u(s) is Skorohod integrable, u(s) G for a.a. s and D t u(s) is Skorohod 
integrable for a.a. t, then 

(2.7) M ( s ) 55 ( s )) = J D t u(s)5B(s) +u(t) for a.a. (t, u), 

where J Q T ip(s)5B(s) denotes the Skorohod integral of ip with respect to £?(•). (See [N], page 
35-38 for a definition of Skorohod integrals and for more details.) 

Some other basic properties of the Malliavin derivative D t are the following: 

(i) Chain rule ([N], page 29) 

Suppose Fi, . . . , F m G Ql\ 2 an d that ip : R m — > R is C with bounded partial deriva- 
tives. Then ijj(F ir - ■ , F m ) G ^i, 2 and 

(2.8) A^(Fi, • • • , F m ) = #(^i> • • • > F m )D t Fi. 

i=i ° Xi 

(ii) Integration by parts/duality formula ([N], page 35) 

Suppose -u(t) is T t — adapted with £7 [J* « 2 (£)dt] < oo and let F G 2$ . Then 



(2.9) E[F I u{t)dB(t)\=E[f u(t)D t Fdt}. 

Jo Jo 



2.2 Malliavin calculus for N(-) 

The construction of a stochastic derivative/Malliavin derivative in the pure jump martingale 
case follows the same lines as in the Brownian motion case. In this case the corresponding 
Wiener-Ito chaos expansion theorem states that any F G L 2 (jF T ,P) (where in this case 
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Tt = F^ is the a— algebra generated by r](s) := J* J Rq zN(dr, dz); < s < t) can be 
written as 

oo 

(2.10) F = Y,Wn); fn e L 2 ((A x V ) n ), 

n=0 

where L 2 ((X x u) n ) is the space of functions f n (ti, z±, . . . ,t n , z n ); U G [0, T], z,i G Mo 
such that f n G L 2 ((\ x z/)") and /„ is symmetric with respect to the pairs of variables 

n j -^n J • 

It is important to note that in this case the n— times iterated integral I n (f„) is taken 
with respect to N(dt,dz) and not with respect to dr)(t). Thus, we define 

(2.11) /„(/„) =7l! / / / / U{tuZ U --- ,t n ,Z n )N{dt U dz 1 )---N{dt n ,dz n ) 



^Rn-'O 



for / n G L 2 ((A x 

The Ito isometry for stochastic integrals with respect to N(dt, dz) then gives the following 
isometry for the chaos expansion: 



'™lli 2 ((Axi/)")- 



(2-12) 11^11!^) = 5> ! l^ 

71=0 

As in the Brownian motion case we use the chaos expansion to define the Malliavin derivative. 
Note that in this case there are two parameters t, z, where t represents time and z ^ 
represents a generic jump size. 

Definition 2.3 (Malliavin derivative D t , z ) [BDL0P], [DM0P] Let &[ N 2 ] be the space 
of all F G L 2 (jF T , P) such that its chaos expansion f)2.10p satisfies 



n\\L 2 ((Xxu) 2 ) < °°- 



(2-13) \\F\\* :=$>n!||/, 

1,2 " 

n=l 

For F G , we define the Malliavin derivative of F at (t, z) (with respect to N(-)), D tz F, 
by 

oo 

(2.14) D ttZ F = J2n>In-i(fn(;t,z)), 

71=1 

where J n _x (/„(-, i, -z)) means that we perform the (n— 1)— times iterated integral with respect 
to iV to the first n — 1 variable pairs (ti,zi),--- , (t n , keeping (t n , z n ) = (t,z) as a 
parameter. 
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In this case we get the isometry. 
(2.15) £[/ / (A^MtkOdt] =y)^ll/n|li a( 

JO JRq n=0 

(Compare with (12.61) .) 

Example 2.4 IfF = / J Rq f(t,z)N(dt,dz) for some deterministic /(t, 2) G L 2 (A x 1/), then 

A,*-F = /(£, 2) for a. a. (t, z). 

More generally, if t/>(s, C) is Skorohod integrable with respect to N(5s,d(), t/>(s, C) G ^12 
for a. a. s, £ and D t>z if)(s,£) is Skorohod integrable for a. a. (t, z), then 



(2.16) A, z ( / / ^(s, C Ws, tZC)) = / / D t;Z iP(s, ()N(5s, d() + u(t, z) for a.a. t, z, 
Jo Jr Jo Jr 

where J Q T J R ip(s, ()N(5s, d() denotes the Skorohod integral of ip with respect to N(-, •). (See 
[DM0P] for a definition of such Skorohod integrals and for more details.) 

The properties of A,* corresponding to the properties (2.8) and (2.9) of A are the 
following: 



(i) Chain rule([I], [DM0P]) Suppose F 1} ■ ■ ■ ,F m G 9$ and that 



is 



continuous and bounded. Then 4>(F X , • • • , F m ) G and 

(2.17) D u <f>{F x , ■ ■ ■ , F m ) = (/.(Fx + D ttt Fx, ...,F m + D t , z F m ) -<f>{F x ,..., F m ). 

(ii) Integration by parts/duality formula [DM0P] Suppose *S?(t, z) is //^-adapted 
and E[Jq J ip 2 (t, z)v(dz)dt\ < 00 and let F G . Then 



(2.18) E 



r I I $(t,z)N(dt,dz) 



E 



V(t,z)D t>z Fv(dz)dt 



3 The stochastic maximum principle 

We now return to Problem 11.11 given in the introduction. We make the following assumptions: 



Assumption 3.1 
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(3.1) The functions b : [0, T] x R x U x Q -> R, cr : [0, T] x R x [/ x Q -> R, / : [0, T] x R x 
[/ x Q — > R and g:lxfl->l are all continuously differential) le (C 1 ) with respect to 
iGl and u G U for each t G [0, T] and a. a. uo G f2. 

(3.2) For all t, r G (0, T), t < r, and all bounded ^—measurable random variables a = a(u) 
the control 

(3 a (s) = a(u) X [t,r](s); s e [0,T\ 

belongs to srfg. 

(3.3) For all u, (3 G =0^ with (3 bounded, there exists 5 > such that 

u + yf3 e for all y G (-5, 8) 

and such that the family 

{^(t,x u+ ye(t),u(t) + yp(t))-[x u+yP (t) 



df 

+ 



(t,x« + nt)xt)+^(tMt)} 



<9« J 

is A x P-uniformly integrable and the family 

d 



{ g '(X u+y(3 {T))— X u+y/3 {T)~j 



dy J ye(-s,s) 

is P-uniformly integrable. 

(3.4) For all u,(3 G st s with (3 bounded the process Y(t) = Y^\t) = ^ (u+2//3) (*)lj/=o exists 
and satisfies the equation 

dY(t) = Y(t~) [£(t, X(t),u(t))dt + ^-(t, X(t),u(t))dB(t) 

r do 

+ / —(t,x(r),u(r),z)N(dt,dz) 

+ (3{r) [— (t, X(t),u(t))dt + -£(t, X(t),u(t))dB(t) 
f 80 

+ / —(t,X(r),u(t-),z)N{dt,dz) 
Y(0) = 0. 

(3.5) For all u G srfg, the following processes 

K(t) := g'(X(T)) + £ g( S , X( S ), «( a ))<fe, 



D t K(t) := D t g'(X(T)) + J D t ^(s, X(s),u(s))ds, 

r T df 

D t , z K(t):=D t , z g'(X(T)) + J D t , z ^(s, X(s),u(s))ds, 
H (s, x, u) := K(s)b(s, x, u) + D s K(s)cr(s, x, u) 
+ / D s z K(s)6(s, x,u, z)u(dz), 

G(t,s) := exp (J {^-(r,X(r),u(r),u,) - \ {^)\r, X(r), u(r), uj)}dr 

■^-(r,X(r),u(r),u)dB(r) 

+ J \ ln ( 1 + ^(r,^(r),u(r),2,w)J - — (r,X(r),u(r),z,u;)ju(dz)dr 

+ jf jf ln(l + ^(r,A;(r-),u(r-),z,w))iV(dr,^)), 

(3.6) 

-^( a ,X( a ),«( S ))G(t,a)(fa, 

(3.7) 

q{t) := D t p{t) , and 

(3.8) 

r(t,z) :=D t , z p(t) 

all exist for < t < s < T, z G K . 

We now define the Hamiltonian for this general problem: 

Definition 3.2 (The general stochastic Hamiltonian) The general stochastic Hamil- 
tonian is the process 

H(t, i,a,u):[0,T]xMx[/xfi^M 

defined by 

H(t, x, u, uj) = f(t, x, u, uj) + p(t)b(t, x, u, uj) + q(t)a(t, x, u, uj) 

(3.9) + / r(t, z)0(t, x,u, z,u)v(dz). 

Remark 3.3 In the classical Markovian case, the Hamiltonian Hi : [0, T] x R x U x R x 
R x TZ -> R is defined by 

(3.10) Hx(t, x,u,p, q,r) = f(t, x,u) + pb(t, x,u) + qa(t, x,u) + / r(z)0(t, x,u, z)u(dz), 

J&n 



where 1Z is the set of functions r : R — > R; see [F0S]. Thus the relation between Hi and H 
is that: 

(3.11) H(t,x,u,u) = i7i(t,x,w,j9(t),g(t),r(t, •)) 
where p(t),q(t) and r(t,z) are given by (3.6)-(3.8). 

We can now formulate our stochastic maximum principle: 
Theorem 3.4 (Maximum Principle) 

(i) Suppose u G srfg is a critical point for J{u), in the sense that 

(3.12) J(m + yP)\ y =o = for all bounded (3 G srfg. 

ay 

Then 

OH 

(3.13) E[— (t,X(t),u(t))\£ t ] = for a.a. t, u, 



where 



X(t)=X {{l) (t), 

H(t, X(t),u) = fit, X{t),u) + p(t)b(t, X(t),u) + q(t)a(t, X(t),u) 
+ / r{t,z)9{t,X(t),u,z)v(dz) , 



with 



p(t)=K(t)+ I —^( s ,X{s),u{s))G{t,s)ds, 



Ox 



and 



{■jfa(r,X(r),u(r),u) - {r,X(r),u(r),u)}dr 

/S Qg. 
—(r,X(r),u(r),cu)dB(r) 

/ s f ( / 09 \ 09 i 

^ { In (l + ^(r, X(r),u(r), z, u)) - ^(r, X(r), u{r), z, u)ju{dz)dr 

+ jT jf In (l + |^(r, X(r-),u(r-), z, uj)N(dr, dz)) , 

K(t) = K<*\t) = g'(X(T)) + j T ^-(s,X(s),u(s))ds. 
(ii) Conversely, suppose there exists u G srfg such that (3.13) holds. Then u satisfies (3.12). 
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Proof. 

(i): Suppose u G sfg is a critical point for J(u). Choose an arbitrary (3 G stg bounded and 
let 5 > be as in (3.3) of Assumption 3.1. 

For simplicity of notation we write u — u, X — X and Y = Y in the following. By (3.3) 
we have 



(3.14) 



where 



=iJ(« + V0)\v=o 



= Eitf{ d / x (t, X(t),u(t))Y(t) + §£(*, X(t),u(W(t)}dt + g'(X(TW(T)}, 



(3.15) 



Y(t) =Y^(t) = jLX^(t)\ y=Q 

= f {§(s,X(s),u(s))Y(s) + f u (s,X(s),u(sW(s)}ds 
+ X(s),u(s))Y(s) + f (s, X(s),u(sW(s)}dB(s) 

+ Jo LM^ X(s),u(s), z)Y(s) + g( a , X( S ), «( a ), z)/?( S )}JV(ds, <fe). 
If we use the short hand notation 

and similarly for f^, I 21 , f^, and f^, we can write 

J Ox ' ou ' ox ' ou ' ox ' ou' 

dY{t) = {f x (t)Y(t) + f u {tm))dt + {%(t)Y(t) + 
( 3 - 16 ) +L if x (t)Y(t) + f u (t)P(t)}N(dt,dz); 

Y(0) = 0. 

By the duality formulas (2.9) and (2.18), we get 

E[g'(X(T))Y(T)} 
= E 



9'(X(T))(^{f x (t)Y(t) + l(t)P(t)}dt 



' L dx v ' V / 



+ 



= E 



/o M*CO)£(0n*) + ^'(x(T))f (t)/3(t) 
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+ D t (g'(X(T)))^ x (t)Y(t) + D t (g'(X(T)))^(t)m 
(3-17) + J Ro [D ttZ (g'(X(T)))i(t)Y(t) + D t , z {g\X{T)))f u {t)mWdz)}dt . 

Similarly we have, using the Fubini theorem, 

= Elfo g(*)(/o{S (s)Y(s) + £(sMs)}ds 
+ Jo LM^ Y ^ + %(sMs)}N(d8, dz))dt\ 

+ L DsA 9 / x (tm(s)Y(s) + f u {s)(3{s)Hdz)}ds)dt 

+ ( fj D s f x {t)dt)[f x {s)Y{s) + ^(s)P(s)} 

+ kdl Ds/£{t)dt)[f x (s)Y(s) + f u (s)P(s)Hdz)}ds]. 

Changing the notation s — > t, this becomes 

SU? %(t)Y(t)dt] = E\^{^f x {s)ds)\f x {t)Y{t) + 
( 3 - 18 ) +(jf D t f x {s)ds)[%(t)Y{t) + 9 f u {tW)\ 

+ k^t D tMs)ds)[f x {t)Y{t) + f (t)/?(t)]^(dz)}dt]. 

Recall 

(3.19) K(t):=g'(X(T)) + [ %{s)ds. 

By combining (3.17)-(3.19), we get 

E[fo{m&)Y(t) + f u (t)f3(t)) 

(3-20) + D t K(t)(%(t)Y(t) + %(t)P(t)) 

+ j Ro D t>z K(t){f x (t)Y{t) + f u (t)f3(t)Hdz) + 9 / u (t)(3(t)}dt] = 0. 

Now apply the above to (3 = f3 a G srfg of the form f3 a (s) = ax[t,t+h](s) , for some t,h E 
(0, T), t + h < T, where a = a (a;) is bounded and ^-measurable. Then Y^ a \s) = for 
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< s < t and hence (3.20) becomes 
(3.21) A 1 + A 2 = 0, 

where 
A 1 = E 

A 2 = E 



rt+h ql Q r an a £ 



Note that, by (3.16), with Y(s) = Y^ a \s) and s > t + h, the process Y(s) follows the 
following dynamics 

(3.22) d Y(s) = Y(s-){^(s)ds+^(s)dB(s) + J ^( s )N(ds, dz)} , 

for s > t + h with initial condition Y(t + h) in time t + h. This equation can be solved 
explicitly and we get 

(3.23) Y(s)=Y(t + h)G(t + h,s); s>t + h, 
where, in general, for s > t, 

G(t, s) = exp ( [ { * (r) - | (!) V)}^ + jf' £(r )d B(r) 
+ j^ a y ln(l + |^(r))#(dr,tfc) 



+ 




In ( 1 + — (r) 

ox 



))-Yx {r M dz)dr )- 



That Y(s) indeed is the solution of (3.22) can be verified by applying the It 6 formula to 
Y(s) given in (3.23). Note that G(t, s) does not depend on h, but Y(s) does. 
Put 

(3.24) H (s, x,u) = K(s)b(s, x,u) + D s K(s)a(s, x,u) + / D SjZ K(s)9(s, x,u, z)v(dz), 

JRn 



and H (s) = H^ u \s) = H (s, X(s),u(s)). Then 
Differentiating with respect to h at h = we get 



s)Y{s)ds 



d 



d 



(3-25) Th M^ = 7,E 



dh 



t+h 



dH 

dx 



(s)Y(s)ds 



J h=o dh 



T 



t+h 



dx 



s)Y(s)ds 



J h=0 
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Since Y(t) = and since dH /dx(s) is cadlag we see that 

" t+h dH 



(3.26) 



dh 



dx 



{s)Y(s)ds 



0. 



J h=0 



Therefore, using (3.23) and that Y(t) = 0, 

T dH 



— A I —F 

dh 1]h =° dh 



J 

J t+h 



dx 



s)Y(t + h)G(t + h,s)ds 



(3.27) 
By (3.16) 



T —E 

dh 
T d 



dh 



E 



dx 
dH 
dx 



s)Y(t + h)G(t + h, s) 



h=0 

ds 



s)G(t,s)Y(t + h) 



h=0 

ds. 



J h=0 



{ — (r)dr + —(r)dB(r) + J — (r)N(dr,dz)j 



f t+fl ( db da f d9 

(3.28) + jf Y(r-){-(r)dr+-(r)dB(r) + -(r)N(dr,dz)}. 

Therefore, by (3.27) and (3.28), 



(3.29) 
where 



d 

dh 



A 1 | fc=0 = A 1 + A 2 , 



Ai = 



d 



-E 



dHn 



(3.30) 
and 



+ 



dh i dx 

de 



[s)G(t, s)a 



t+h 



db 



da 



,r)dr + — — (r)dB(r) 
du du 



du 



r)N(dr, dz) 



ds 



A, 



d 

dh 



E 



J h=0 



(3.31) 



+ 



dx 



-te(s)G(t,s)J t Y(r-){-(r)dr + -(r)dB(i 
*)N(dr, dz)} 



By the duality formulae (2.9), (2.18) we have 



" T d 

- 1 Tk E 



+ 



r +h r db da 

df) ~\ i 

— (r)D r!Z F(t,s)u(dz)\dr\ ds 

du > J h=0 
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(3.32) = j~ E[a{^(t)F(t,s) + ^(t)D t F(t,s) + 
where we have put 
(3.33) 

Since Y(t) =0 we see that 



da 



86 
du 



(t)D t , z F(t,s)v(dz)} 



ds, 



d M 

F(t,s) = ^(s)G(t,s). 



(3.34) 

We conclude that 
d 



A 2 = 0. 



dh A ^ = ^ 



(3.35) 



jT E [a{F(t, s)^(t) + D t F(t, s)^(t) 



du 



du 



D 



d9 ~) 

t , z F(t,s) — (t)u(dz)j 



ds. 



Moreover, we see directly that 



U^ = E[ a {K( t & t) + D t K(At) + / D^kA)^) + fit)} 



du J Rq ' du du 
Therefore, differentiating (3.21) with respect to h at h — gives the equation 



E 



a 



(3.36) 



{ (K(t) + jT F(t, s)ds)^(t) + A (K(t) + jT F(t, s)ds)^(t) 
+ L ^(^) + / T F(M). s )£(t M ^ + |(t))}] =0. 



We can reformulate this as follows: If we define, as in (3.6), 

" T r T dH 

F(t,s)ds = K(t)+ I " 

then (3.36) can be written 



(3.37) p(t) = K(t) + J F(t, s)ds = K(t) + J 



dx 



[s)G(t,s)ds, 



E {/(* , X(t),u) + p(t)b(t, X(t),u) + D t p(t)a(t, X(t),u) 
+ I D ttZ p(t)9(t,X(t),u,z)u(dz)} a 

Jr J u=u(t) . 

Since this holds for all bounded ^-measurable random variable a, we conclude that 



E 



[ ^-H(t,X(t),u) u=u(t) | ^ 



= 0, 



which is (3.13). This completes the proof of (i). 
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(ii): Conversely, suppose (3.13) holds for some u G srfg. Then by reversing the above 
argument we get that (3.21) holds for all f3 a G srfg of the form 

P a (s,w) = a(u)x(t,t+h](s) 

for some t,h G [0, T] with t + h < T and some bounded ^-measurable a. Hence (3.21) 
holds for all linear combinations of such (3 a . Since all bounded f3 G srfg can be approximated 
pointwise boundedly in (t,u) by such linear combinations, it follows that (3.21) holds for 
all bounded (3 G srfg. Hence, by reversing the remaining part of the argument above, we 
conclude that (3.12) holds. 

□ 

4 Applications 

In this section we illustrate the maximum principle by looking at some examples. 

Example 4.1 (Optimal dividend/harvesting rate) 
Suppose the cash flow X(t) = X^ c \t) at time t is given by 

dX(t) = (po(t, u) + bi(t, u)X(t) - c(t))dt + (a (t, cu) + a x (t, u)X{t))dB{t) 

+ J no (9 (t,z,uj)+9 1 (t,z,uj)X(t))N(dt,dz)] 

X(0) = x G R, 

where 

b (t) = b Q (t,u), b^t) = 6i(t,w) : [0,T] x D, ^ R 

a {t) = <To(t,u), <Ji(t) = <Ji(t,u) : [0,T] x Q h-> R and 

^ (t,^) = 9 (t,z,u), 0i(t,z) = 9!(t,z,u) : [0,T]xRxHkR 

are given ^-predictable processes. 

Here c(t) > is our control (the dividend/harvesting rate), assumed to belong to a family 
Ag of admissible controls, contained in the set of ^-predictable controls. 

Suppose the performance functional has the form 

r r T 

(4.2) J(c) = E 



[ z(s)u(c(8))d8 + c* (c) cn 

Jo 
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where U : [0, +00] 1— > R is a C 1 utility function, £(s) = £(s,u) is an jF r predictable process 
and ( = C{uj) is an jF T -measurable random variable. 
We want to find c G Ag such that 



(4.3) 



sup J(c) = J(c). 

c&Ag 



Using the notation from the previous section, we note that in this case we have, with 
c — u, 

f(t,x,c) = £(t)U(c) and g{x) = (x. 

Hence 

K{t) = J^(s,X(s),c(s))ds + g'(X(T))=C, 

H (t,x,c) = C(bo(t) + bi(t)x-c)+D t C(<r (t)+a 1 (t)x)+ [ D t>z t(6 (t, z) + 9 1 (t,z)x)u(dz) 

Jr 

G(t,s) = ex p(^" {bi(r) - ±<rl(r)} dr + J <7i(r)dS(r) 



+ 



In (1 + Bi{r,z)) - Bx(r,z) \v{dz)dr + / / In (l + ^(r, z) ]JV(dr, dz) ) . 



Then 



(4.4) p(*) = C+ [ T ((b 1 (r) + D r (a 1 (r)+ [ D r>z ( 9 1 (r, z)u(dz)) G(t,r) dr , 
and the Hamiltonian becomes 

(4.5) H(t,x,c) = Z(t)U(c)+p(t)(b (t) + b 1 (t)x-c) + D t p(t) (a (t) + ax(t)x) 

+ [ D t>z p(t) (6 (t, z) + 9 1 {t, z)x)v(dz) 



Hence, if c G Ag is optimal for the problem ( 14. 3D . we have 

d 



= E 



dc 



H(t,X(t),c(t))\£ t 



E[{fflU'(c(t))-p(t)}\&] 

u'(c(t))E[am]-E[ P (m]. 



We have proved: 



Theorem 4.2 If there exists an optimal dividend/harvesting rate c(t) > for problem ff^W , 
then it satisfies the equation 

(4.6) U\c{t))E[i{t)\£ t ]=E\p{t)\£ t ], 
where p(t) is given by (14. 4p . 
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Example 4.3 (Optimal portfolio) 

Suppose we have a financial market with the following two investment possibilities: 

(i) A risk free asset, where the unit price So(t) at time t is given by 

( 4 - 7 ) dS (t) = Pt S (t)dt- S (0) = 1; te[0,T\. 

(ii) A risky asset, where the unit price Si(t) at time t is given by 

dSxit) = 5i(r) a t dt + (3 t dB(t) + f R ((t, z)N(dt,dz) ; t G [0,T] 
(4.8) L ° J 

5i(0) > 0. 

Here p t , a t , f3 t and ((t, z) are bounded .^-predictable processes; t G [0, T], 2 6 M an d T > 
is a given constant. We also assume that 

C(*> z ) — ~ 1 a - s - f° r a - a - z 
and T 



|log(l + C(t,^))| ^(^)^ <oo. 

'0 JR ^ 

A portfolio in this market is an £ r predictable process u(t) representing the amount invested 
in the risky asset at time t. When the portfolio u(-) is chosen, the corresponding wealth 
process X(t) = X^ u \t) satisfies the equation 

dX(t) = [p t X(t) + (a t - (h)u(t)]dt + p t u(t)dB(t) 
(4.9) +[ ((t,z)u(t-)N(dt,dz)] X(0)=x>0. 

The partial information optimal portfolio problem is to find the portfolio u G As which 
maximizes 

J(u) = E[U(X^\T),u)] 

where U(x) — U(x, ou) : R x Q — > R is a given jF r measurable random variable for each x and 
x — > U(x,u) is a utility function for each u. We assume that x — > C/(x) is C l and C/'(x) is 
strictly decreasing. The set As of admissible portfolios is contained in the set of ^-adapted 
portfolios u(t) such that (4.9) has a unique strong solution. 

With the notation of the previous section we see that in this case we have 

/(*, x, u) = f{t } x, u, uj) = 0, g(x, u) = U (x, u), 
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Thus 

(4.10) 

and 



b(t, x, u) = p t x + (a t - p t )u, a(t, x, u) = f3 t u, 
9(t, x, u, z) = ({t, z)u. 

K(t) = U'(X(T)) = K, 

H (t, x, u) = K(p t x + (a t - p t )u) + D t K(3 t u 
+ I D t , z K((t,z)uN(dt,dz), 

G(t, s) = exp (^j^ p r dr^j . 

(4.12) p(t) = U'(X(T)) + jf" Kp s exp ^ p r d^j ds , 

and the Hamiltonian becomes 



and 

(4.11) 

Thus 



Hit, x, u) = pit) [p t x + (a t - p t )u] + D t p(t)(5 t u 

+ I D ttg p(t)C(t,z)uu{dz). 



(4.13) 



By the maximum principle, we now get the following condition for an optimal control. 
Theorem 4.4 Ifu{t) is an optimal control with corresponding X (t) , p(t) then 



E 



d_ 

du 



H(t,X(t),u) u= u(t) I St 







i.e. 



(4.14) E \p(t)(a t - Pt ) + PtD t p(t) + / D t , z p(t)C(t, z) v{dz) \ S t ] = 0. 

Equation (4.14) is an interesting new type of equation. We could call it a Malliavin differ- 
ential type equation in the unknown process pit). Note that if we can find p(t), then we also 
know U'(X(T)) and hence X(T). In particular, we see that the optimal final wealth fulfills 
the following Malliavin differential type equation in the unknown random variable U'(X(T)) 



E 



U'(X(T))(a T - p T ) + (3 T D T U 



'(X(T)) + f 



D T , z U'iXiT))aT,z)vidz)\S T \ =0. 
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In this paper we will not discuss general solution methods of this type of Malliavin differ- 
ential equations, but leave this issue for future research. Instead we complete by considering 
a solution in the special case when 

(4.15) v = p t = 0, \/3t\ > 5 > and £ t = T t \ 0<t<T, 
where 5 > is a given constant. Then (4.14) simplifies to 

(4.16) a t E[K\F t ] + PtE[D t K\F t ] = . 
By the Clark-Ocone theorem we have 

K = E[K]+ [ E[D t K\T t ]dB{t), 
Jo 

which implies that 

(4.17) E[K\F t ] = E[K\ + f E[D s K\F s \dB{s). 

Jo 

Define 

(4.18) M t := E[K\T t ] = E[U' {X {T))\T t \. 
Then by substituting (4.16) into (4.17) we get 



M t = E[K\ - [ ^M s dB(s) 
Jo ys 

or 

dM t = -%M t dB t 

Pt 

which has the solution 

(4.19) M t = E[U'(X(T))} exp ( - J* ^dB(s) - \ J* {^) ds 

This determines U'(X(T)) = M T = K modulo the constant E[U'(X(T))} = M . 

Given K the corresponding optimal portfolio u is given as the solution of the backward 
stochastic differential equation 



(4.20) 



dX(t) = a t u(t)dt + /3tu(t)dB(t); t < T 
X(T) = {U')-\K) 
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This equation can be written 



(4.21) 



dX(t) = Ptu(t)dB(t); t < T 
X(T) = (Ur 1 (K), 

where 
(4.22) 

which is a Brownian motion with respect to the probability measure Q defined by 



a t 



dB(t) = -£dt + dB(t), 
Pt 



(4.23) 



where 



dQ = N T dP on T T , 



! -a n 2 



ds | . 



By the Clark-Ocone theorem under change of measure [KO] we have 



(4.24) X(T)=E Q [X(T)]+ / Eq 

Jo 

Comparing (4.21) and (4.24) we get 
(4.25) 

Using Bayes' rule we conclude 



D t X{T)-X{T) J D t (^>)dB(s)) \F t ]dB(t) 



Ht) = jE Q [(D t X{T) - X(T) J D t (j-)dB(s)) \ T t 



Theorem 4.5 Suppose u e Ar is an optimal portfolio for the problem 
(4.26) 

with 



sup E[U(X<- u \T),u)] 



dX (u) (t) = a t u(t)dt + p t u(t)dB(t). 



Then 
(4.27) 
and 



u(t) 



PtN t 



-E 



X T (D,X(T)-X(T) I D,(^)dB(a)) \T t 



(4.28) X(T) = (UT l (M T ), 

where M t is given by (4-19). 
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Corollary 4.6 Suppose 



U(x) = -x^F(u) 

7 



for some Tt -measurable bounded F. Then 



(4.29) 
where 
(4.30) 



u(t) = X(t) 



1 E[N T (D t Y -Y £ D t {f-)dB{s)) \ T t \ 



E[N T Y | T t \ 



Y 



Proof. In this case we get 



and 

X(t) = E Q [X(T)\F t ] 
Therefore the result follows from (4.27). 



AY/') (^) = ur r r 



MJ- 1 E[N T Y\F t ] 



N, 



□ 
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